Spark: Refactor SparkReadConf use primitive type for confs with default values#7429
Conversation
szehon-ho
left a comment
There was a problem hiding this comment.
Mostly looks good to me. But wanted to double check, is it a backward compatible change?
User will ever get NoSuchMethodError?
Under what case would user have this error? @szehon-ho |
|
we can get the values in both long, Long, so using the function call directly would not be problematic but let's say if some one is extending SparkReadConf and over-riding the Would like to know your thoughts and make the changes accordingly, as |
|
Yea I am thinking, if our caller does not recompile their library on newer Iceberg version , they will hit it, but I think its not the normal flow. Thanks for double checking. |
|
Thanks, @singhpk234! |
|
Thanks @singhpk234 ! i was thinkiing about @szehon-ho point , I don't think we can set an expectation that people can extend SparkReadConf and have signatures align. So overall, this change looks good to me |
Co-authored-by: Prashant Singh <psinghvk@amazon.com>
About Change
Address review feedback #7422 (comment) .
cc @aokolnychyi